Real-Time Grasp Detection Using Convolutional Neural Networks

We present an accurate, real-time approach to robotic grasp detection based on convolutional neural networks. Our network performs single-stage regression to graspable bounding boxes without using standard sliding window or region proposal techniques. The model outperforms state-of-the-art approaches by 14 percentage points and runs at 13 frames per second on a GPU. Our network can simultaneously perform classification so that in a single step it recognizes the object and finds a good grasp rectangle. A modification to this model predicts multiple grasps per object by using a locally constrained prediction mechanism. The locally constrained model performs significantly better, especially on objects that can be grasped in a variety of ways.Comment: Accepted to ICRA 201

Upwelling to Outflowing Oxygen Ions at Auroral Latitudes during Quiet Times: Exploiting a New Satellite Database

The mechanisms by which thermal O+ escapes from the top of the ionosphere and into the magnetosphere are not fully understood even with 30 years of active research. This thesis introduces a new database, builds a simulation framework around a thermospheric model and exploits these tools to gain new insights into the study of O+ ion outflows. A dynamic auroral boundary identification system is developed using Defense Meteorological Satellite Program (DMSP) spacecraft observations at 850 km to build a database characterizing the oxygen source region. This database resolves the ambiguity of the expansion and contraction of the auroral zone. Mining this new dataset, new understanding is revealed. We describe the statistical trajectory of the cleft ion fountain return flows over the polar cap as a function of activity and the orientation of the interplanetary magnetic field y-component. A substantial peak in upward moving O+ in the morning hours is discovered. Using published high altitude data we demonstrate that between 850 and 6000 km altitude, O+ is energized predominantly through transverse heating; and acceleration in this altitude region is relatively more important in the cusp than at midnight. We compare data with a thermospheric model to study the effects of solar irradiance, electron precipitation and neutral wind on the distribution of upward O+ at auroral latitudes. EUV irradiance is shown to play a dominant role in establishing a dawn-focused source population of upwelling O+ that is responsible for a pre-noon feature in escaping O+ fluxes. This feature has been corroborated by observations on platforms including the Dynamics Explorer 1 (DE-1), Polar, and Fast Auroral Snapshot SnapshoT (FAST) spacecraft. During quiet times our analysis shows that the neutral wind is more important than electron precipitation in establishing the dayside O+ upwelling distribution. Electron precipitation is found to play a relatively modest role in controlling dayside, and a critical role in controlling nightside, upwelling O+. This thesis provides a new database, and insights into the study of oxygen ion outflows during quiet times. These results and tools will be essential for researchers working on topics involving magnetosphere-ionosphere interactions

Fast Deep Matting for Portrait Animation on Mobile Phone

Image matting plays an important role in image and video editing. However, the formulation of image matting is inherently ill-posed. Traditional methods usually employ interaction to deal with the image matting problem with trimaps and strokes, and cannot run on the mobile phone in real-time. In this paper, we propose a real-time automatic deep matting approach for mobile devices. By leveraging the densely connected blocks and the dilated convolution, a light full convolutional network is designed to predict a coarse binary mask for portrait images. And a feathering block, which is edge-preserving and matting adaptive, is further developed to learn the guided filter and transform the binary mask into alpha matte. Finally, an automatic portrait animation system based on fast deep matting is built on mobile devices, which does not need any interaction and can realize real-time matting with 15 fps. The experiments show that the proposed approach achieves comparable results with the state-of-the-art matting solvers.Comment: ACM Multimedia Conference (MM) 2017 camera-read

A Self-Supervised Feature Map Augmentation (FMA) Loss and Combined Augmentations Finetuning to Efficiently Improve the Robustness of CNNs

Deep neural networks are often not robust to semantically-irrelevant changes in the input. In this work we address the issue of robustness of state-of-the-art deep convolutional neural networks (CNNs) against commonly occurring distortions in the input such as photometric changes, or the addition of blur and noise. These changes in the input are often accounted for during training in the form of data augmentation. We have two major contributions: First, we propose a new regularization loss called feature-map augmentation (FMA) loss which can be used during finetuning to make a model robust to several distortions in the input. Second, we propose a new combined augmentations (CA) finetuning strategy, that results in a single model that is robust to several augmentation types at the same time in a data-efficient manner. We use the CA strategy to improve an existing state-of-the-art method called stability training (ST). Using CA, on an image classification task with distorted images, we achieve an accuracy improvement of on average 8.94% with FMA and 8.86% with ST absolute on CIFAR-10 and 8.04% with FMA and 8.27% with ST absolute on ImageNet, compared to 1.98% and 2.12%, respectively, with the well known data augmentation method, while keeping the clean baseline performance.Comment: Accepted at ACM CSCS 2020 (8 pages, 4 figures

The Next Generation of Human-Drone Partnerships: Co-Designing an Emergency Response System

The use of semi-autonomous Unmanned Aerial Vehicles (UAV) to support emergency response scenarios, such as fire surveillance and search and rescue, offers the potential for huge societal benefits. However, designing an effective solution in this complex domain represents a "wicked design" problem, requiring a careful balance between trade-offs associated with drone autonomy versus human control, mission functionality versus safety, and the diverse needs of different stakeholders. This paper focuses on designing for situational awareness (SA) using a scenario-driven, participatory design process. We developed SA cards describing six common design-problems, known as SA demons, and three new demons of importance to our domain. We then used these SA cards to equip domain experts with SA knowledge so that they could more fully engage in the design process. We designed a potentially reusable solution for achieving SA in multi-stakeholder, multi-UAV, emergency response applications.Comment: 10 Pages, 5 Figures, 2 Tables. This article is publishing in CHI202